Machine learning for prediction of acute kidney injury in patients diagnosed with sepsis in critical care

Background and objective Acute Kidney Injury (AKI) is a common and severe complication in patients diagnosed with sepsis. It is associated with higher mortality rates, prolonged hospital stays, increased utilization of medical resources, and financial burden on patients’ families. This study aimed to establish and validate predictive models using machine learning algorithms to accurately predict the occurrence of AKI in patients diagnosed with sepsis. Methods This retrospective study utilized real observational data from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. It included patients aged 18 to 90 years diagnosed with sepsis who were admitted to the ICU for the first time and had hospital stays exceeding 48 hours. Predictive models, employing various machine learning algorithms including Light Gradient Boosting Machine (LightGBM), EXtreme Gradient Boosting (XGBoost), Random Forest (RF), Decision Tree (DT), Artificial Neural Network (ANN), Support Vector Machine (SVM), and Logistic Regression (LR), were developed. The dataset was randomly divided into training and test sets at a ratio of 4:1. Results A total of 10,575 sepsis patients were included in the analysis, of whom 8,575 (81.1%) developed AKI during hospitalization. A selection of 47 variables was utilized for model construction. The models derived from LightGBM, XGBoost, RF, DT, ANN, SVM, and LR achieved AUCs of 0.801, 0.773, 0.772, 0.737, 0.720, 0.765, and 0.776, respectively. Among these models, LightGBM demonstrated the most superior predictive performance. Conclusions These machine learning models offer valuable predictive capabilities for identifying AKI in patients diagnosed with sepsis. The LightGBM model, with its superior predictive capability, could aid clinicians in early identification of high-risk patients.


Introduction
Sepsis, a prevalent critical condition in clinical practice [1][2][3], poses a significant risk for the development of Acute Kidney Injury (AKI) [4].The kidneys are particularly vulnerable to reduced blood perfusion and certain treatment interventions, such as aggressive fluid resuscitation and mechanical ventilation, commonly employed in patients diagnosed with sepsis.
Currently, treatment for sepsis combined with AKI is mainly non-specific and lacks preventive measures.Studies have shown that the mortality rate of sepsis patients with AKI can increase from 38.2% to 70% [5,6].[7] reported that the incidence of AKI in sepsis patients is 40% to 50%, and the mortality rate after AKI increases 6-8 times, which is 7.79 times higher than non-sepsis patients.AKI is a common complication in ICU patients, with sepsis accounting for about 60% of cases, and 25% of patients requiring Continuous Renal Replacement Therapy (CRRT) treatment, resulting in longer hospital stays, higher mortality rates, and increased financial burden on families [8][9][10].However, early identification and treatment of sepsis patients with AKI can promote early renal recovery, shorten hospital stays, and improve survival rates [11,12].Unfortunately, identifying high-risk AKI patients in the ICU is challenging for clinicians.Therefore, there is an urgent need to develop and promote reliable predictive models to identify these patients early and provide them with timely and effective interventions.
Despite advancements in medical treatments for AKI, the mortality rate associated with this condition remains unchanged [13].Sepsis patients with AKI often experience multiple organ failure, microvascular dysfunction, and systemic inflammatory response syndrome, which further complicates clinical management [14][15][16].However, early and effective interventions can potentially reverse AKI and reduce associated mortality rates [17].Therefore, the identification of high-risk AKI patients among ICU patients diagnosed with sepsis is crucial.Improving early identification and preventive measures for AKI holds significant importance in enhancing the clinical outcomes of patients.Predicting AKI in sepsis patients has been a widely discussed topic in the field of critical care medicine [18].
Currently, numerous researchers are seeking widely applicable early AKI prediction methods.Several biomarkers have been reported to be associated with AKI in sepsis, including procalcitonin [19], microRNA-22-3p [20], neutrophil gelatinase-related lipoproteins [21], urine miR-26b [22], soluble thrombomodulin [23], Tissue Inhibitor of Metalloproteinase-2(TIMP-2), and Insulin-like Growth factor-Binding Protein 7 (IGBP-7) [24].However, the high requirements and costs associated with these biomarker detection technologies hinder their clinical applicability.Several scoring systems, such as the Simplified Acute Physiology Score (SAPS-II), the Acute Physiology and Chronic Health Score II (APACHE-II), and the Sequential Organ Failure Assessment (SOFA), are also utilized for AKI prediction.However, these scoring systems have shown poor specificity and sensitivity in predicting AKI in patients diagnosed with sepsis, leading to unsatisfactory results [25,26].
To address the limitations mentioned above, researchers have proposed the use of multivariate predictive models based on traditional statistical methods to predict the progression of AKI in sepsis patients.For instance, [27] developed a predictive model for AKI in sepsis patients using Logistic Regression (LR) and COX proportional hazard model.While this model demonstrated good prediction accuracy, LR assumes linear correlations between independent and dependent variables, which may oversimplify the complex nonlinear relationships and lead to decreased model performance.Therefore, there is a need to explore more efficient and accurate predictive tools for managing sepsis patients.With advancements in computer technology and statistical theory, machine learning has gained attention from clinical practitioners as a potential solution for this purpose [28].
The LR model assumes a linear relationship between features and outputs, while machine learning algorithms like Light Gradient Boosting Machine (LightGBM), Extreme Gradient Boosting (XGBoost), Support Vector Machines (SVM), Artificial Neural Networks (ANN), Decision Trees (DT), and Random Forests (RF) offer the capacity for automatic learning from data [29].These machine learning models can capture non-linear correlations and interactions, allowing for the exploration of more complex relationships between features through intricate model structures and algorithms, making them more flexible [30].Therefore, machine learning algorithms present a promising opportunity to enhance the prediction of AKI in septic patients.
Machine learning technology has emerged as a promising approach for disease prediction models.Within the realm of AKI prediction, researchers have diligently explored the potential of machine learning algorithms.For instance, [31] developed an automated continuous random prediction algorithm for AKI, showcasing its efficacy in early identification of high-risk patients.Additionally, [32] conducted a comparative study among SVM, ANN, RF, and Simplified Acute Physiology Score II (SAPSII) in predicting AKI patient mortality, where RF exhibited superior potential.[33] extracted patient data from the Medical Information Mart for Intensive Care III (MIMIC-III) database, specifically focusing on patients diagnosed with Congestive Heart Failure (CHF).Several common machine learning classifiers were compared to select the optimal predictive model for identifying CHF patients at high risk of developing AKI.Among these classifiers, the LightGBM model demonstrated the best predictive performance, achieving an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.80313.However, the optimal machine learning model for predicting AKI in septic patients remains unclear.This study aims to develop and validate an AKI predictive model using machine learning technologies and compare the performance of different models to determine the most effective one.

Data source
The sepsis patient data utilized in this study originated from Medical Information Mart for Intensive Care IV (MIMIC-IV) version 2.0 [34], which were launched on June 12, 2022, as a publicly accessible database from a single center.MIMIC-IV is a publicly available relational dataset set up by the Laboratory for Computational Physiology that includes deidentified health data related to thousands of intensive care unit admissions, including real hospital stays for patients enrolled to a tertiary academic medical center in Boston, MA, USA.MIMIC-IV includes comprehensive data for every patient while they are in the hospital: laboratory measurements, medications managed, vital signs documented.Extra data available contains the out-of-hospital date of death, data from the online medical record system (which provides for height and weight), and more information for continuous infusions in the ICU.The dataset is extensively adopted by inquirers and engineers in the world, so as to promote studies in clinical informatics, epidemiology, and machine learning.
Ethical statement.This research utilized data from the MIMIC-IV database.Given that the MIMIC IV database consists of de-identified data and is publicly accessible, the study was not subject to specific ethical review.However, we conducted the research in accordance with fundamental ethical principles and ensured the validity and fairness of the study process.

Participants
When an infection occurred and patients' SOFA score � 2, the patients qualification were taken into account.The occurrence of AKI among patients diagnosed with sepsis during admission were assessed using the Kidney Disease Improving Global Outcomes (KDIGO) standards [35].Patients who stayed in the ICU less than 48 hours, aged < 18 years old and > 90 years old, or multiple admissions other than the first admission old were excluded.Additionally, patients who received CRRT and had a history of severe chronic kidney disease on individual clinical outcomes were also excluded.Furthermore, variables with more than 20% missing values were excluded to minimize bias resulting from missing data.Missing values were handled using median imputation, a method that involves replacing missing values with representative measures of central tendency to maintain the general distributional shape and characteristics of the original data.The final cohort consisted of 10,575 patients, of whom 8,575 developed AKI, while 2,000 did not.Within this cohort, 1,747 patients experienced septic shock, 13 individuals faced severe hypoxemia, and 9,960 required ventilation.On average, the time between the onset of sepsis and ICU admission was 1.04 days.

Variable selection
With the help of PostgreSQL 13, we installed the database on the service, and then extracted the demographic, clinical characteristics, etc., of sepsis patients according to the corresponding SQL codes.The MIMIC-IV database provided patient information in the initial 48 hours after admission.The data below were adopted in this research: 1. Demographic features, including age, gender and race;   (2) EFB Technique for LightGBM.The EFB algorithm employs a graph-based approach, treating each characteristic as a node within the structure.It connects unique features based on their relationships, forming bundled feature sets within the graph.The greedy strategy for the EFB characteristic bundling involves the following steps:

Prediction model
1. Using the features as vertices in the graph, connect unique features (samples existing nonzero simultaneously), with the number of coexisting non-zero samples as the edge weight.
2. Sort the features based on vertex degree, indicating higher degrees represent greater conflicts among characteristics (less likely to bind with other features).
3. Establish a maximum conflict threshold, K.

Study population
In the final cohort, 10,575 patients were included, of which 8,575 cases (81.1%) developed AKI during hospitalization in the MIMIC-IV database.A total of 47 variables were selected for model construction.To determine the predictive power of different features, the collections and distributions of categorical variables were first analyzed by performing a chi-squared test (p < 0.05).Table 2 lists the statistical values of the features of the detailed patient demographic data with their corresponding definitions.

Model performance comparisons
Experimental comparisons were conducted on a 64-bit Windows 10 system equipped with Python 3.11 (www.python.org).The computer features an Intel(R) Core (TM) i5-6300U CPU running at 2.40GHz and 16.00 GB RAM.Regarding the predictive model construction, 80% of the cases were randomly allocated to the training set, while the remaining 20% of cases were reserved for the testing set.First, the outcomes of these 7 machine learning approaches are shown in Table 3.The LightGBM model exhibited superior performance compared to other models, achieving the highest scores in accuracy (0.829), F1 score (0.902), and AUC (0.801).Accuracy measures the proportion of correctly predicted samples, the F1 score combines precision and recall, while AUC assesses the model's ability to classify positive and negative samples.Across these metrics, LightGBM consistently outperformed other models, indicating its superior overall performance and stronger predictive capabilities in this classification task, ultimately leading to more accurate sample classification.On the other hand, SVM exhibits the highest specificity and precision, indicating its capability to accurately classify negative instances and correctly predict positive instances, respectively.Meanwhile, DT shows the highest sensitivity, excelling in the correct identification of true positive instances.For the prediction of AKI, it is premised on the need for accurate diagnosis of AKI in patients diagnosed with sepsis.Therefore, sensitivity ought to be taken into consideration.Although the DT model achieved highest sensitivity (0.987), it obtained least precision (0.816).To take into account both precision and recall, F1 score providing a balance between precision and recall is regarded as a metric.Overall, each model showcases distinct strengths, with LightGBM leading in overall performance, while SVM and DT excel in certain specific metrics.When moving towards real-world applications, it would be prudent to factor in specific requirements and focal points.
Then, the evaluation then proceeds to assess and explore the models' performance concerning the AUC.As depicted in Fig 2, the Receiver Operating Characteristic (ROC) curve illustrates the predictive models' performance.Finally, an analysis of feature importance in the LightGBM model was conducted, revealing the top-ranking features in descending order.As illustrated in Fig 4, the most critical predictors included first day urine output, OASIS, first care unit, Pao2 and glucose, which contribute to higher predictive powers than the bottom features.This indicates that these features contribute significantly to the prediction results.In the next section of the discussion, a more indepth analysis of the AKI diagnosis problem will be conducted, potentially uncovering new correlations and patterns.

Discussion
Sepsis, a prevalent critical condition in the ICU, stands as a primary cause of mortality among critically ill patients globally.Within the realm of critical care medicine, sepsis remains a challenging and extensively studied subject.Its intersection with acute kidney injury (AKI) presents as a common clinical complication in patients diagnosed with sepsis.Timely detection and proactive early active treatment can significantly improve the survival rate of AKI patients.Hence, the development and promotion of responsible AKI predictive models for sepsis patients is of great significance in improving patient prognosis.Based on the experimental findings, the following conclusions can be drawn: Firstly, from the Table 3, it can be observed that the LightGBM model's superior performance, boasting the highest AUC (0.801), accuracy (0.829), and F1 score (0.902) in predicting AKI among patients diagnosed with sepsis.This robust performance indicates its exceptional capability in discriminating between positive and negative instances.Such findings suggest the model's effectiveness in accurately ranking samples and allocating higher probabilities to positive cases, thereby enhancing decision-making processes.However, it's noteworthy that SVM exhibits exceptional specificity and precision.Researchers and practitioners might consider selecting models based on specific task requirements.
Moreover, the discussion encompasses 10,575 critically ill patients diagnosed with sepsis, among whom 8,575 experienced AKI during their hospitalization.This incidence rate of AKI among the patients studied herein notably surpasses the reported 40%-50% in relevant literature.Several factors may contribute to this disparity: (1) The study being a retrospective realworld investigation in a single center, where disease incidence is influenced by the center's diagnosis and treatment standards; (2) The inclusion criteria encompass all sepsis patients, irrespective of AKI status or clinical staging.Experimental outcomes demonstrate the LightGBM model's exceptional predictive probability calibration, closely mirroring actual probabilities.Even in the face of an imbalanced distribution between positive and negative Secondly, from Fig 4, it can be seen that the top 5 important features were first day urine output, OASIS, first care unit, Pao2 and glucose, which contribute to higher predictive powers than the bottom features.Among these features, urine volume, is the most significant index for predicting AKI, conforming to the suggestions of the KDIGO guidelines.A decrease in urine output can be an early sign of AKI.OASIS is a scoring system that evaluates the severity of illness in patients, which can provide valuable information about the overall health status of a patient, including the presence or risk of AKI.The location or type of care unit where a patient is admitted can be an important factor in identifying AKI patients.Certain care units, such as ICU, are more likely to have patients at higher risk of developing AKI due to the severity of their illness or the nature of the treatments they receive.Pao2 reflects the oxygenation status of the blood.In patients with AKI, there may be reduced renal blood flow and impaired oxygen delivery to the kidneys.This can result in tissue hypoxia and further damage to the kidneys.Low Pao2 levels can indicate the presence of respiratory dysfunction or lung disease, which can contribute to the development or progression of AKI.Abnormal glucose levels, especially high levels (hyperglycemia), can contribute to the development or worsening of AKI.Hyperglycemia can cause inflammation, oxidative stress, and endothelial dysfunction, all of which can damage the kidneys.Monitoring and assessing these five important features can help in the early identification of AKI patients.
Thirdly, LightGBM offers a feature importance ranking, aiding in the identification of the most impactful features and providing insights into the model's predictions.This functionality empowers clinicians to recognize factors contributing to AKI prediction, enabling them to prioritize interventions.Adjusting medication dosages, optimizing fluid management, or implementing renal protective strategies based on identified risk factors can significantly influence patient outcomes.
Furthermore, machine learning algorithms like LightGBM demonstrate significant capability in managing extensive datasets and enabling real-time information processing, rendering them highly compatible with integration into electronic health record (EHR) systems.This integration harbors the potential to establish an automated system capable of predicting AKI risk and generating alerts, thereby providing clinicians with timely notifications and empowering proactive patient care.
Nevertheless, it's crucial to recognize several limitations in our study.The retrospective observational design introduces the potential for selection bias.The dataset used was sourced from a single center in the United States, casting doubt on the generalizability of our findings to a broader population.Thus, extensive large-scale, multi-center studies are warranted to confirm the model's suitability across diverse settings.Furthermore, further research and validation in real-world clinical environments are essential to comprehensively evaluate its practical impact and feasibility for widespread adoption.
Fig 1 shows the flow chart of patients' selection.

Fig 1 .
Fig 1. Flow chart of patients' selection.https://doi.org/10.1371/journal.pone.0301014.g001 Fig 2  demonstrates that LightGBM exhibits a notably higher AUC compared to the other models.While LR, XGBoost, SVM, and RF's ROC curves demonstrate better performance than DT and ANN, they still fall short of LightGBM's performance.Among these seven models, LightGBM's ROC curve attains the highest point, indicating superior classification performance.Specifically, this model achieves the highest AUC, signifying a favorable balance between sensitivity (True Positive Rate, TPR) and specificity (True Negative Rate, TNR), achieving a higher true positive rate while maintaining a lower false positive rate.Furthermore, Fig3depicts the confusion matrices of the various models, providing additional insights.This suggests that the LightGBM model excels in detecting positive samples, showcasing improved sensitivity-an essential aspect for early AKI patient identification.In summary, it evidences superior performance in predicting AKI patients.
Different data instances have varying impacts on data gain calculation.Instances with larger gradients contribute more to data gain.GOSS eliminates instances with smaller gradients while retaining those with larger gradients to maintain data gain estimation accuracy, resulting in more precise information gain estimates with smaller datasets.In GOSS, prediction values (preds) and LOSS are computed.Training examples are sorted based on their absolute values in descending order.Assuming constant factors a and b, we retain the top x × 100% instances to form the subset 'topSet', and sample the smaller gradient instances at y × 100% to create the subset 'randSet'.It's crucial to amplify the gradient of the small gradient sample by (1 − x)/y.Algorithm 1 outlines this process.Algorithm 1 Gradient-based One-Side Sampling.
This research compared several machine learning models, namely LightGBM, XGBoost, RF, DT, ANN, SVM, and LR.Both LightGBM and XGBoost, as boosting algorithms, offer distinct advantages in scenarios with limited training samples, short training times, and limited reference knowledge.Among these models, LightGBM exhibited superior performance, a point we will delve into further.LightGBM, an open-source gradient boosting framework, extends upon the gradient boosting framework of DT.This framework enhances model efficiency and reduces memory usage.LightGBM finds common usage in ranking, classification, and various machine learning tasks.It integrates two key technologies: Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB), enabling faster algorithm execution while maintaining high precision.(1)GOSS Technique for LightGBM.
Begin an outer loop to iterate through the sorted features.For each feature, traverse existing feature bundles.If the feature fits within a cluster, add it.If not, create a new cluster and add the feature.
Input: C: characteristic, M: max conflict count